Skip to content

Run pytest on Ubuntu, macOS, and Windows in CI#293

Merged
wpak-ai merged 4 commits into
cppalliance:developfrom
leostar0412:ci/test-ubuntu-macos-windows
Jun 16, 2026
Merged

Run pytest on Ubuntu, macOS, and Windows in CI#293
wpak-ai merged 4 commits into
cppalliance:developfrom
leostar0412:ci/test-ubuntu-macos-windows

Conversation

@leostar0412

@leostar0412 leostar0412 commented Jun 16, 2026

Copy link
Copy Markdown
Collaborator

Summary

CI previously ran the full pytest suite only on ubuntu-latest, while the README documents macOS and Windows support. This PR splits the single test job into three parallel jobs (test-ubuntu, test-macos, test-windows) so path handling, dependency install behavior, and platform-specific code paths are exercised before merge.
Changes:

  • Add .github/actions/run-pytest composite action (checkout, uv install, DB verify, pytest with --cov-fail-under=90, OS-suffixed artifacts).
  • test-ubuntu: existing Docker services.postgres + apt (pandoc, leveldb).
  • test-macos / test-windows: ikalnytskyi/action-setup-postgres@v8 (Postgres 16) + OS-specific pandoc/leveldb setup.
  • Add scripts/verify_postgres_connection.py (replaces bash heredoc; works on Windows PowerShell).
  • Preserve plyvel Windows exclusion in lock files: plyvel==1.5.1 ; sys_platform != "win32".
  • Update README, CONTRIBUTING, and docs/Celery_test.md to describe the new CI layout.
    lint, pyright, and compose-smoke remain Ubuntu-only. No continue-on-error on any test leg.
    Follow-up after merge: update branch protection required checks from test to test-ubuntu, test-macos, and test-windows.

Apps touched

  • (none — CI, docs, and dependency locks only)

Test plan

  • python -m pytest (or scoped: python -m pytest <app>/tests) — full suite passed locally on macOS (3218 passed, 4 skipped, 90.42% coverage)
  • uv run pyright (if typed code changed) — N/A (no application code changes)
  • lint-imports (if imports or cross-app coupling changed) — N/A
  • App command smoke-tested (if collector/command changed):
  • CI green on all three test jobs: test-ubuntu, test-macos, test-windows

Docs / coupling

  • cross-app-dependencies.md updated (if FKs or cross-app imports changed)
  • python scripts/generate_service_docs.py run (if services.py or core/protocols.py changed)
  • App README or docs/ updated (if behavior or ops changed)

Closes #289

Summary by CodeRabbit

  • Tests
    • Split CI test runs into separate Ubuntu, macOS, and Windows jobs using PostgreSQL.
    • Added a shared CI test runner that sets up Python, verifies PostgreSQL connectivity, runs pytest with coverage (HTML/XML), enforces a 90% minimum, and uploads test/coverage artifacts.
  • Documentation
    • Updated README and contributing guidance to reflect the new multi-job CI layout and platform-specific test expectations.
    • Expanded Windows Celery worker troubleshooting with current CI context.
  • Chores
    • Refreshed dependency constraints/CVE notes and lock-file guidance (including plyvel marker verification).

@coderabbitai

coderabbitai Bot commented Jun 16, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 432ad66f-b8bf-4677-a6ef-42e869b5b8de

📥 Commits

Reviewing files that changed from the base of the PR and between 2a911ff and 4a5d71c.

📒 Files selected for processing (2)
  • README.md
  • docs/CODEOWNERS_and_branch_protection.md
✅ Files skipped from review due to trivial changes (2)
  • docs/CODEOWNERS_and_branch_protection.md
  • README.md

📝 Walkthrough

Walkthrough

This PR extracts pytest execution into a reusable GitHub composite action, replaces the single CI test job with Ubuntu/macOS/Windows platform-specific jobs, adds a PostgreSQL verification script, updates repository docs to describe the new cross-platform test workflow and platform notes, and bumps core dependency versions for security.

Changes

Cross-platform CI matrix and workflow

Layer / File(s) Summary
Shared pytest runner and DB check
.github/actions/run-pytest/action.yml, scripts/verify_postgres_connection.py, requirements.in
Adds a reusable composite action that installs the test environment under Python 3.13 using uv, verifies PostgreSQL connectivity with a Django-aware helper script, runs pytest with coverage (HTML and XML), enforces --cov-fail-under=90, outputs junit.xml, and conditionally uploads artifacts. A requirements note preserves the Windows exclusion marker for plyvel when recompiling lock files.
Platform-specific workflow jobs
.github/workflows/actions.yml
Replaces the single test flow with test-ubuntu, test-macos, and test-windows, each installing platform dependencies (APT, Homebrew, Chocolatey), setting up PostgreSQL where needed (service containers on Ubuntu, ikalnytskyi/action-setup-postgres on macOS/Windows), and delegating test execution to the shared action with OS-specific inputs.
Documentation alignment
CONTRIBUTING.md, README.md, docs/Celery_test.md, docs/CODEOWNERS_and_branch_protection.md
Updates CI documentation to reference the three test-* jobs, restates shared test environment details and coverage enforcement, adds a Windows Celery note linking to the Windows CI job for context on the solo pool behavior, and documents branch protection status checks with required and optional CI job names.
Dependency constraint updates
requirements.in
Updates core dependencies: cryptography is constrained to >=48.0.1,<49 and aiohttp is bumped from >=3.14.0,<4 to >=3.14.1,<4, with revised CVE-fix documentation.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

Suggested reviewers

  • snowfox1003
  • wpak-ai
  • jonathanMLDev

Poem

🐇 I hopped through three lanes of CI light,
Ubuntu, macOS, Windows in sight.
A postgres check gave a confident grin,
Then pytest bounded merrily in.
Coverage carrots were bundled away,
And docs now point the proper way.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The PR title clearly and concisely summarizes the main change: introducing pytest execution across three platforms (Ubuntu, macOS, Windows) in CI, which aligns with the primary objective of the changeset.
Description check ✅ Passed The PR description comprehensively covers the Summary, Apps touched, Test plan, and Docs/coupling sections per the template, with clear details on changes made, test results, and documentation updates.
Linked Issues check ✅ Passed All acceptance criteria from issue #289 are met: CI matrix implemented for three platforms, test suite passes with platform-aware skips, platform-conditional dependencies preserved, no continue-on-error flags, and documentation updated to reflect actual CI coverage.
Out of Scope Changes check ✅ Passed All changes are within scope of issue #289: CI workflow restructuring, composite action creation, cross-platform test infrastructure, dependency lock file updates, and documentation adjustments—no unrelated code modifications.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
.github/workflows/actions.yml (1)

90-159: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Set explicit least-privilege permissions for the new test jobs.

These jobs currently inherit default token scopes. Pinning to read-only where possible reduces CI token blast radius.

🔒 Suggested hardening
   test-ubuntu:
+    permissions:
+      contents: read
     runs-on: ubuntu-latest
     timeout-minutes: 15
@@
   test-macos:
+    permissions:
+      contents: read
     runs-on: macos-latest
     timeout-minutes: 25
@@
   test-windows:
+    permissions:
+      contents: read
     runs-on: windows-latest
     timeout-minutes: 25
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/actions.yml around lines 90 - 159, Add explicit
least-privilege `permissions` configuration to each of the three test jobs:
test-ubuntu, test-macos, and test-windows. These jobs currently inherit default
token scopes which poses a security risk. Set the `permissions` field for each
job to an empty object or specify only the minimal permissions needed (typically
none for test-only jobs). This reduces the CI token blast radius by explicitly
restricting what each job can do with its GitHub token.

Source: Linters/SAST tools

🧹 Nitpick comments (1)
requirements.in (1)

6-6: ⚡ Quick win

Automate the plyvel marker check instead of relying on manual verification.

This safeguard is important for Windows legs, so it should be enforced by CI (or pre-commit) rather than a comment-only reminder.

♻️ Minimal guard you can add (example CI check)
+      - name: Verify lockfiles keep Windows marker for plyvel
+        run: |
+          rg -n 'plyvel.*sys_platform != "win32"' requirements.lock requirements-dev.lock
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@requirements.in` at line 6, The manual verification comment for ensuring
plyvel maintains the `; sys_platform != "win32"` marker in lock files should be
replaced with an automated check. Create a CI or pre-commit script that verifies
both lock files contain the plyvel dependency with the correct platform marker
for Windows exclusion. This ensures the Windows build safeguard is enforced
automatically rather than relying on manual developer verification of the
comment reminder.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@README.md`:
- Around line 175-186: The README claims "A failure on any test job blocks
merge" but the branch protection rules do not actually enforce required status
checks for the test-ubuntu, test-macos, or test-windows jobs. Fix this
discrepancy by either: (1) configuring the three test jobs as required status
checks in GitHub's branch protection settings for both develop and main
branches, or (2) revising the final sentence in the CI section of README.md to
clarify that test failures should block merge once required checks are
configured, rather than stating they currently do. Choose the approach that
aligns with your team's policy and implement accordingly.

---

Outside diff comments:
In @.github/workflows/actions.yml:
- Around line 90-159: Add explicit least-privilege `permissions` configuration
to each of the three test jobs: test-ubuntu, test-macos, and test-windows. These
jobs currently inherit default token scopes which poses a security risk. Set the
`permissions` field for each job to an empty object or specify only the minimal
permissions needed (typically none for test-only jobs). This reduces the CI
token blast radius by explicitly restricting what each job can do with its
GitHub token.

---

Nitpick comments:
In `@requirements.in`:
- Line 6: The manual verification comment for ensuring plyvel maintains the `;
sys_platform != "win32"` marker in lock files should be replaced with an
automated check. Create a CI or pre-commit script that verifies both lock files
contain the plyvel dependency with the correct platform marker for Windows
exclusion. This ensures the Windows build safeguard is enforced automatically
rather than relying on manual developer verification of the comment reminder.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 30e84736-c13e-4104-bbff-8544f07b96d5

📥 Commits

Reviewing files that changed from the base of the PR and between c16c95b and 9083ccd.

⛔ Files ignored due to path filters (2)
  • requirements-dev.lock is excluded by !**/*.lock
  • requirements.lock is excluded by !**/*.lock
📒 Files selected for processing (7)
  • .github/actions/run-pytest/action.yml
  • .github/workflows/actions.yml
  • CONTRIBUTING.md
  • README.md
  • docs/Celery_test.md
  • requirements.in
  • scripts/verify_postgres_connection.py

Comment thread README.md Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/actions.yml:
- Around line 112-115: The workflow file contains three instances of the
actions/checkout action (at lines 112-115, 133-136, and 157-160) that lack the
persist-credentials: false setting. This allows the workflow token to be written
to local git config, creating unnecessary token-exposure surface for test jobs
that don't require push authentication. Add persist-credentials: false to the
with section of the actions/checkout action at all three locations to disable
credential persistence and improve security.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 32771549-e30d-4606-a007-64562c18afbe

📥 Commits

Reviewing files that changed from the base of the PR and between 9083ccd and 62d9e57.

📒 Files selected for processing (2)
  • .github/actions/run-pytest/action.yml
  • .github/workflows/actions.yml
💤 Files with no reviewable changes (1)
  • .github/actions/run-pytest/action.yml

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Inline review comments failed to post. This is likely due to GitHub's internal server error or limits when posting large numbers of comments. If you are seeing this consistently it is likely a permissions issue. Please check "Moderation" -> "Code review limits" under your organization settings.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/actions.yml:
- Around line 112-115: The workflow file contains three instances of the
actions/checkout action (at lines 112-115, 133-136, and 157-160) that lack the
persist-credentials: false setting. This allows the workflow token to be written
to local git config, creating unnecessary token-exposure surface for test jobs
that don't require push authentication. Add persist-credentials: false to the
with section of the actions/checkout action at all three locations to disable
credential persistence and improve security.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 32771549-e30d-4606-a007-64562c18afbe

📥 Commits

Reviewing files that changed from the base of the PR and between 9083ccd and 62d9e57.

📒 Files selected for processing (2)
  • .github/actions/run-pytest/action.yml
  • .github/workflows/actions.yml
💤 Files with no reviewable changes (1)
  • .github/actions/run-pytest/action.yml
🛑 Comments failed to post (1)
.github/workflows/actions.yml (1)

112-115: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Disable persisted checkout credentials in all test jobs.

Line 112, Line 133, and Line 157 use actions/checkout without persist-credentials: false, so the workflow token is written to local git config by default. These are test jobs and don’t need push auth, so this is avoidable token-exposure surface.

🔐 Suggested hardening diff
       - name: Checkout
         uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
         with:
           fetch-depth: 0
+          persist-credentials: false

Also applies to: 133-136, 157-160

🧰 Tools
🪛 zizmor (1.25.2)

[warning] 112-115: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false

(artipacked)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/actions.yml around lines 112 - 115, The workflow file
contains three instances of the actions/checkout action (at lines 112-115,
133-136, and 157-160) that lack the persist-credentials: false setting. This
allows the workflow token to be written to local git config, creating
unnecessary token-exposure surface for test jobs that don't require push
authentication. Add persist-credentials: false to the with section of the
actions/checkout action at all three locations to disable credential persistence
and improve security.

Source: Linters/SAST tools

…1 for security fixes; adjust requirements files accordingly. Enhance CI workflow to set CPPFLAGS and LDFLAGS for leveldb installation.
@leostar0412

Copy link
Copy Markdown
Collaborator Author

@coderabbitai full review

@coderabbitai

coderabbitai Bot commented Jun 16, 2026

Copy link
Copy Markdown
✅ Action performed

Full review finished.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
.github/workflows/actions.yml (1)

90-177: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Harden token exposure for test jobs.

The workflow runs test code without explicit least-privilege token permissions, and checkout keeps credentials in git config by default. Tighten both to reduce credential abuse risk during CI execution.

🔧 Suggested hardening patch
 name: CI
 
 on:
@@
 concurrency:
   group: ${{ github.workflow }}-${{ github.ref }}
   cancel-in-progress: true
+
+permissions:
+  contents: read
 
 jobs:
@@
       - name: Checkout
         uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
         with:
           fetch-depth: 0
+          persist-credentials: false
@@
       - name: Checkout
         uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
         with:
           fetch-depth: 0
+          persist-credentials: false
@@
       - name: Checkout
         uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
         with:
           fetch-depth: 0
+          persist-credentials: false
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/actions.yml around lines 90 - 177, Add explicit
least-privilege token permissions to restrict credential exposure in the test
job workflows. For each of the three test jobs (test-ubuntu, test-macos,
test-windows), add a permissions block at the job level with only the minimum
required permissions (typically empty or just contents: read for checkout).
Additionally, update each Checkout step using actions/checkout to disable
credential persistence by adding with: persist-credentials: false, which
prevents git credentials from being stored in the git config and reduces the
attack surface if the checkout token is exposed during test execution.

Source: Linters/SAST tools

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In @.github/workflows/actions.yml:
- Around line 90-177: Add explicit least-privilege token permissions to restrict
credential exposure in the test job workflows. For each of the three test jobs
(test-ubuntu, test-macos, test-windows), add a permissions block at the job
level with only the minimum required permissions (typically empty or just
contents: read for checkout). Additionally, update each Checkout step using
actions/checkout to disable credential persistence by adding with:
persist-credentials: false, which prevents git credentials from being stored in
the git config and reduces the attack surface if the checkout token is exposed
during test execution.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 4d1b76c4-1403-4d24-9a1b-df5352b3e348

📥 Commits

Reviewing files that changed from the base of the PR and between c16c95b and 2a911ff.

⛔ Files ignored due to path filters (2)
  • requirements-dev.lock is excluded by !**/*.lock
  • requirements.lock is excluded by !**/*.lock
📒 Files selected for processing (7)
  • .github/actions/run-pytest/action.yml
  • .github/workflows/actions.yml
  • CONTRIBUTING.md
  • README.md
  • docs/Celery_test.md
  • requirements.in
  • scripts/verify_postgres_connection.py

…ements and enable required status checks for CI jobs on protected branches.

@jonathanMLDev jonathanMLDev left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm

@wpak-ai wpak-ai merged commit 9490d69 into cppalliance:develop Jun 16, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cross-Platform CI Matrix: Add macOS + Windows

3 participants